Guiding method and guiding system

ABSTRACT

A commodity layout database retains commodity arrangement information in which a plurality of kinds of commodity identification information for identifying a plurality of commodities arranged indoors and indoor arrangement positions of the plurality of commodities are associated with each other. A photographing unit photographs the periphery of an indoor. A commodity specifying unit analyzes a photographed image and specifies commodity identification information included in the photographed image. A guidance information creating unit specifies an indoor arrangement position of a registered commodity registered in advance by the user on the basis of the specified commodity identification information, the commodity arrangement information, and registered commodity identification information for identifying the registered commodity, and creates guidance information for guiding the user to the specified arrangement position of the registered commodity. A display unit presents the created guidance information.

TECHNICAL FIELD

The present invention relates to a guiding method and a guising systemfor guiding a user to the position of a commodity in which the user isinterested.

BACKGROUND ART

A GPS (Global Positioning System) is a system for specifying a positionon the Earth. The GPS is effective when the position of an outdoormobile body is specified in the order of several meters, for example,when a user drives an automobile on a road. A car navigation systemspecifies the present position of the user (the automobile) using theGPS and guides the user to a destination.

For example, Patent Literature 1 discloses a shopping guidanceinformation providing apparatus that searches for a store present withina fixed distance from a terminal position and selling a registeredpurchase planned commodity and, when the store selling the purchaseplanned commodity is found near a user, informs, for guidance, a mobileterminal held by the user of information concerning the store.

However, in an indoor environment such as the inside of a building, insome case, a GPS signal is blocked by a roof or a wall and cannot bereceived. In some case, the GPS signal cannot be directly receivedbehind a building and the GPS signal arriving bypassing the building isreceived to specify the present position. In this case, detectionaccuracy of the present position is sometimes deteriorated by theinfluence of the GPS signal arriving bypassing the building. In theconventional GPS, the present position cannot be specified in the orderof 10 centimeters to 1 meter. Therefore, there is a problem in that itis difficult to specify the present position of a user in an indoorenvironment such as the inside of a store and guide the user to theposition of a predetermined commodity.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Unexamined Patent Publication No.2002-175465

SUMMARY OF INVENTION

The present invention has been devised to solve the problem and it is anobject of the present invention to provide a guiding method and aguiding system that can guide a user to an indoor position of acommodity in which the user is interested.

A guiding method according to an aspect of the present inventionincludes: a commodity arrangement information retaining step ofretaining, in a commodity arrangement information retaining unit,commodity arrangement information in which a plurality of kinds ofcommodity identification information for identifying a plurality ofcommodities arranged indoors and indoor arrangement positions of theplurality of commodities are associated with each other; a photographingstep of photographing the periphery of an indoor user; a specifying stepof analyzing a photographed image photographed in the photographing stepand specifying commodity identification information included in thephotographed image; and a guiding step of specifying an indoorarrangement position of a registered commodity registered in advance bythe user on the basis of the commodity identification informationspecified in the specifying step, the commodity arrangement informationretained in the commodity arrangement information retaining unit, andregistered commodity identification information for identifying theregistered commodity, and presenting guidance information for guidingthe user to the specified arrangement position of the registeredcommodity.

According to the present invention, it is possible to guide the user toan indoor position of a commodity in which the user is interested usingarrangement positions of commodities included in a photographed imageand an indoor arrangement position of a registered commodity registeredin advance by a user.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing the overall configuration of a guidingsystem in an embodiment of the present invention.

FIG. 2 is a diagram showing the configurations of a guidance serviceapparatus, a guiding apparatus, and an appliance in the embodiment ofthe present invention.

FIG. 3 is a flowchart for explaining the operations of the appliance andthe guidance service apparatus in the embodiment of the presentinvention.

FIG. 4 is a first flowchart for explaining the operations of the guidingapparatus and the guidance service apparatus in the embodiment of thepresent invention.

FIG. 5 is a second flowchart for explaining the operations of theguiding apparatus and the guidance service apparatus in the embodimentof the present invention.

FIG. 6 is a diagram for explaining a photographing direction in which auser photographs the inside of a store using the guiding apparatus inthe embodiment of the present invention.

FIG. 7 is a diagram showing an example of an in-store layout diagramretained in a commodity layout database.

FIG. 8 is a diagram showing an example of commodity layout informationretained in the commodity layout database.

FIG. 9 is a diagram showing an example of guidance information displayedwhen a commodity of interest is present in a photographed image.

FIG. 10 is a diagram showing another example of the guidance informationdisplayed when the commodity of interest is present in the photographedimage.

FIG. 11 is a diagram showing an example of a user interface componentimage used in creating the guidance information shown in FIG. 10.

FIG. 12 is a diagram showing an example of guidance informationdisplayed when the commodity of interest is absent in the photographedimage.

FIG. 13 is a diagram showing another example of the guidance informationdisplayed when the commodity of interest is absent in the photographedimage.

FIG. 14 is a diagram showing an example of a user interface componentimage used in creating the guidance information shown in FIG. 13.

FIG. 15 is a diagram showing an example of guidance informationdisplayed when the commodity of interest is present in an invisibleposition in the photographed image.

FIG. 16 is a diagram showing an example of a user interface componentimage used in creating the guidance information shown in FIG. 15.

FIG. 17 is a diagram showing an example of guidance informationdisplayed when a specified commodity is not a commodity of interest andthe commodity of interest is present in a photographed image.

FIG. 18 is a diagram showing an example of a user interface componentimage used in creating the guidance information shown in FIG. 17.

FIG. 19 is a diagram showing an example of guidance informationdisplayed when a commodity of interest is absent in a photographed imagein which commodities are two-dimensionally arranged.

FIG. 20 is a diagram showing an example of a user interface componentimage used in creating the guidance information shown in FIG. 19.

DESCRIPTION OF EMBODIMENTS

An embodiment of the present invention is explained below with referenceto the drawings. Note that the embodiment explained below is an embodiedexample of the present invention and does not limit the technical scopeof the present invention.

A guiding system in this embodiment is a system that guides a user tothe position of a commodity in which the user is interested (a targetcommodity that the user desires to purchase) in a store in which a largenumber of commodities are arranged on display shelves. The user holds aportable terminal (a smart phone) having a camera function. The guidingsystem guides the user to a target position using an image photographedby the user using a camera.

In this embodiment, the following five examples are explained.

(1) In-Store Search 1

First, an application photographs the inside of a store using the cameramounted on the smart phone. Subsequently, the application or a serverperforms, through image recognition, matching of a commodity listshowing a commodity in which the user is interested and a photographedimage. Subsequently, the application indicates on a screen of the smartphone that the commodity in which the user is interested is present inthe vicinity.

(2) In-Store Search 2

First, the application photographs the inside of the store using thecamera mounted on the smart phone. Subsequently, the application or theserver performs, through image recognition, matching of a commodity listshowing a commodity in which the user is interested and a photographedimage. Subsequently, the application superimposes and presents, on aphotographed image displayed on the screen of the smart phone, a markindicating the position of the commodity in which the user isinterested.

(3) In-Store Guidance 1

First, the application photographs the inside of the store using thecamera mounted on the smart phone. Subsequently, the application or theserver specifies displayed commodities through image recognition andestimates a position of the smart phone (the user) from commodity layoutinformation in which the commodities and positions where the commoditiesare arranged are associated with each other. Subsequently, theapplication presents, from the estimated position of the smart phone andthe position of the commodity in which the user is interested, arelative position of the commodity in which the user is interested onthe screen of the smart phone as a text or the like or notifies therelative position by sound.

(4) In-Store Guidance 2

First, the application photographs the inside of the store using thecamera mounted on the smart phone. Subsequently, the application or theserver specifies displayed commodities through image recognition andestimates a position of the smart phone (the user) from commodity layoutinformation in which the commodities and positions where the commoditiesare arranged are associated with each other. Subsequently, theapplication presents, from the estimated position of the smart phone andthe position of the commodity in which the user is interested, arelative direction to the commodity in which the user is interested onthe screen of the smart phone.

(5) In-Store Guidance 3

First, the application photographs the inside of the store using thecamera mounted on the smart phone. Subsequently, the application or theserver specifies displayed commodities through image recognition andestimates a position of the smart phone (the user) from commodity layoutinformation in which the commodities and positions where the commoditiesare arranged are associated with each other. Subsequently, theapplication superimposes and presents, on a photographed image displayedon the screen of the smart phone, a mark indicating the position of thecommodity in which the user is interested present on the rear side of ashelf.

<Overall Configuration>

FIG. 1 is a diagram showing the overall configuration of the guidingsystem in the embodiment of the present invention. The guiding systemincludes a guidance service apparatus 100, a guiding apparatus 200, andan appliance 300.

The appliance 300 is a home electric appliance such as a refrigerator, awashing machine, or a microwave oven.

The appliance 300 is used when a user registers a commodity in which theuser is interested, in the guidance service apparatus 100. For example,a user interface of the appliance 300 displays a commodity list andreceives selection of a commodity from the commodity list by the user.When the commodity is selected, the appliance 300 notifies the guidanceservice apparatus 100 of the selected commodity.

The guidance service apparatus 100 is, for example, a server apparatus.The guidance service apparatus 100 stores the notified commodity in acommodity-of-interest list as a commodity in which the user isinterested (hereinafter referred to as commodity of interest). Thecommodity in which the user is interested is, for example, a commoditythat the user explicitly designates and desires to purchase (a shoppingnote). The commodity in which the user is interested is a commodityautomatically selected by an appliance in a home such as a refrigeratoror a microwave oven as a commodity needed to be purchased soon byautomatically managing and grasping a use state or a storage state ofpurchased commodities in the refrigerator.

The guiding apparatus 200 is, for example, a smart phone that executesan application. The guiding apparatus 200 may be a portable tabletcomputer or a cellular phone or may be a terminal having a camerafunction. The guiding apparatus 200 downloads a guidance applicationfrom the guidance service apparatus 100. The guidance application ismanaged by an application managing unit included in the guidance serviceapparatus 100. The application managing unit transmits the guidanceapplication according to a request from the guiding apparatus 200.

Note that the guiding apparatus 200 may download the guidanceapplication from a server apparatus that is different from the guidanceservice apparatus 100 and manages the guidance application. Note thatthe guiding apparatus 200 and the appliance 300 may be the sameapparatus.

The guiding apparatus 200 executes the downloaded guidance application.The guidance application (program) executed on the guiding apparatus 200controls a camera provided in the guiding apparatus 200 and photographsan image of the periphery of the user. The guiding apparatus 200transmits the photographed image to the guidance service apparatus 100.The guidance service apparatus 100 analyzes the received image andspecifies a photographed commodity. The guiding apparatus 200 may retainin advance analysis data used for an analysis of an image. In this case,the guidance application analyzes the photographed image and specifiesthe photographed commodity. When the guiding apparatus 200 specifies thecommodity included in the photographed image, the guiding apparatus 200notifies the guidance service apparatus 100 of the specified commodity.

As a method of specifying the commodity included in the photographedimage, for example, there is a method of recording and managing, in adatabase, a package photograph of a commodity or a text described on apackage of a commodity and performing matching of an image with thepackage photograph or the text. Matching of a text extracted from thephotographed image and a text recorded in the database in advance may beperformed to specify the commodity included in the photographed image.Note that the commodity included in the photographed image may bespecified by other methods.

Subsequently, the respective configurations of the guidance serviceapparatus 100, the guiding apparatus 200, and the appliance 300 areexplained.

FIG. 2 is a diagram showing the configurations of the guidance serviceapparatus 100, the guiding apparatus 200, and the appliance 300 in theembodiment of the present invention. The guidance service apparatus 100and the guiding apparatus 200 are communicably connected to each othervia a network. The guidance service apparatus 100 and the appliance 300are communicably connected to each other via the network. The networkis, for example, the Internet.

The appliance 300 includes a commodity-of-interest input receiving unit301, a display unit 302, and a communication unit 303.

The commodity-of-interest input receiving unit 301 receives an input ofa commodity of interest by the user. The commodity-of-interest inputreceiving unit 301 receives the input of the commodity of interest bythe user, for example, through a keyboard displayed on a screen of atouch panel. The commodity-of-interest input receiving unit 301 mayreceive the input of the commodity of interest by the user through soundrecognition.

The display unit 302 displays various kinds of information. The displayunit 302 displays selectable commodities. The display unit 302 displayscommodities used in the appliance 300. For example, when the appliance300 is a refrigerator, the display unit 302 displays foodstuffs such asfrozen foods, vegetables, and fruits. The user selects a desiredcommodity of interest out of a commodity list displayed on the displayunit 302.

The communication unit 303 receives various kinds of information fromthe guidance service apparatus 100 and transmits various kinds ofinformation to the guidance service apparatus 100. The communicationunit 303 receives a selectable commodity list from the guidance serviceapparatus 100. The communication unit 303 transmitscommodity-of-interest information concerning the commodity of interestreceived by the commodity-of-interest input receiving unit 301 to theguidance service apparatus 100. Note that the commodity-of-interestinformation includes a user ID for identifying the user.

The guiding apparatus 200 includes a photographing unit 201, a presentposition acquiring unit 202, a display unit 203, and a communicationunit 204.

The photographing unit 201 photographs the periphery of the user in abuilding and acquires a photographed image. Note that the photographedimage may be a still image or may be a moving image.

The present position acquiring unit 202 is configured by, for example, aGPS and acquires the present position of the guiding apparatus 200.

The communication unit 204 receives various kinds of information fromthe guidance service apparatus 100 and transmits various kinds ofinformation to the guidance service apparatus 100. The communicationunit 204 transmits the photographed image photographed by thephotographing unit to the guidance service apparatus 100. Thecommunication unit 204 receives guidance information transmitted by theguidance service apparatus 100. The guidance information is informationfor guiding the user to the position of a commodity of interest.

The display unit 203 displays various kinds of information. The displayunit 203 displays the guidance information received by the communicationunit 204.

The guidance service apparatus 100 includes a commodity layout database101, a commodity-of-interest database 102, a commodity specifying unit103, a guidance information creating unit 104, an updating unit 105, anda communication unit 106.

The commodity layout database 101 retains commodity arrangementinformation in which a plurality of kinds of commodity identificationinformation for identifying a plurality of commodities arranged in thebuilding and arrangement positions in the building of the plurality ofcommodities are associated with each other. Note that the commodityarrangement information is present for each store. The commodityidentification information includes, for example, commodity names orimages of the commodities.

The commodity-of-interest database 102 retains commodity-of-interestidentification information for identifying a commodity of interestregistered in advance by the user. The commodity-of-interest database102 retains a commodity-of-interest list for each user and, for example,retains the commodity-of-interest list in association with the user IDfor identifying the user. The commodity-of-interest identificationinformation includes, for example, a commodity name of the commodity ofinterest or an image of the commodity of interest.

The commodity specifying unit 103 analyzes the photographed imagephotographed by the photographing unit 201 of the guiding apparatus 200and specifies commodity identification information included in thephotographed image.

The guidance information creating unit 104 specifies an arrangementposition in the building of the commodity of interest on the basis ofthe commodity identification information specified by the commodityspecifying unit 103, the commodity arrangement information retained inthe commodity layout database 101, and the commodity-of-interestidentification information for identifying the commodity of interestregistered in advance by the user and creates guidance information forguiding the user to the specified arrangement position of the commodityof interest. The guidance information creating unit 104 specifies, onthe basis of present position information received from the guidingapparatus 200, a store where the user is currently present and reads outcommodity arrangement information corresponding to the specified storefrom the commodity layout database 101.

When the commodity identification information specified by the commodityspecifying unit 103 and the commodity-of-interest identificationinformation retained in the commodity-of-interest database 102 coincidewith each other, the guidance information creating unit 104 creates, asthe guidance information, an image indicating a position in thephotographed image of the commodity of interest. That is, when thecommodity identification information specified by the commodityspecifying unit 103 and the commodity-of-interest identificationinformation retained in the commodity-of-interest database 102 coincidewith each other, this means that the commodity of interest is present inthe photographed image. The guidance information is informationindicating where in the photographed image the commodity of interest ispresent.

When the commodity identification information specified by the commodityspecifying unit 103 and the commodity-of-interest identificationinformation retained in the commodity-of-interest database 102 do notcoincide with each other, the guidance information creating unit 104creates, as the guidance information, an image indicating the positionof the commodity of interest. That is, when the commodity identificationinformation specified by the commodity specifying unit 103 and thecommodity-of-interest identification information retained in thecommodity-of-interest database 102 do not coincide with each other, thismeans that the commodity of interest is absent in the photographedimage. The guidance information is information indicating where on theoutside of the photographed image the commodity of interest is present.

Note that, when the commodity identification information specified bythe commodity specifying unit 103 and the commodity-of-interestidentification information retained in the commodity-of-interestdatabase 102 do not coincide with each other, the guidance informationcreating unit 104 may create, as the guidance information, an imageindicating a direction in which the commodity of interest is present.

When the commodity identification information specified by the commodityspecifying unit 103 and the commodity-of-interest identificationinformation retained in the commodity-of-interest database 102 do notcoincide with each other and the commodity of interest is present in ahidden position in the photographed image, the guidance informationcreating unit 104 creates, as the guidance information, an imageindicating an imaginary position of the commodity of interest. Forexample, when the commodity of interest is not displayed on a shelfincluded in the photographed image and the commodity of interest isdisplayed on a shelf on the rear side of the shelf included in thephotographed image, the guidance information creating unit 104 createsguidance information imaginarily indicating the position of thecommodity of interest not seen from the present position of the user.

The updating unit 105 updates, on the basis of the photographed imagephotographed by the photographing unit 201 of the guiding apparatus 200,the commodity arrangement information retained in the commodity layoutdatabase 101. The updating unit 105 specifies an arrangement position inthe building of the commodity identification information specified bythe commodity specifying unit 103. When the specified arrangementposition is different from an arrangement position of the commodityidentification information included in the commodity arrangementinformation, the updating unit 105 updates the commodity arrangementinformation to change the arrangement position of the commodityidentification information included in the commodity arrangementinformation to the specified arrangement position.

The communication unit 106 receives various kinds of information fromthe appliance 300 and transmits various kinds of information to theappliance 300. The communication unit 106 receives various kinds ofinformation from the guiding apparatus 200 and transmits various kindsof information to the guiding apparatus 200. The communication unit 106receives the present position information and the photographed imageinformation transmitted by the guiding apparatus 200. The communicationunit 106 receives the commodity-of-interest information transmitted bythe appliance 300. The communication unit 106 transmits the guidanceinformation created by the guidance information creating unit 104 to theguiding apparatus 200.

Subsequently, the operation of the guiding system in the embodiment ofthe present invention is explained.

FIG. 3 is a flowchart for explaining the operations of the appliance andthe guidance service apparatus in the embodiment of the presentinvention.

First, in step S1, the commodity-of-interest input receiving unit 301 ofthe appliance 300 receives an input of commodity-of-interest informationby the user. The user inputs a commodity that the user is interested in(a commodity of interest). For example, the user inputs, as thecommodity of interest, a commodity that the user plans to purchase. Notethat the commodity-of-interest input receiving unit 301 may receiveselection of a desired commodity out of a plurality of commoditiesdisplayed on the display unit 302. The commodity-of-interest inputreceiving unit 301 may receive an input of a commodity name of thecommodity of interest or a serial number of the commodity of interest.Further, the commodity-of-interest input receiving unit 301 may receivea type of the commodity as the commodity-of-interest information. Thetype of the commodity is not information capable of uniquely identifyingthe commodity such as a specific commodity name but is informationcapable of specifying a rough type of the commodity such asconfectionary, frozen food, meat, fish, or vegetable.

Subsequently, in step S2, the communication unit 303 transmits thecommodity-of-interest information received by the commodity-of-interestinput receiving unit 301 to the guidance service apparatus 100. In thiscase, the communication unit 303 adds the user ID for identifying theuser to the commodity-of-interest information and transmits the user ID.The user ID may be input by the user or may be stored in the appliance300 in advance.

Subsequently, in step S3, the communication unit 106 of the guidanceservice apparatus 100 receives the commodity-of-interest informationtransmitted by the appliance 300.

Subsequently, in step S4, the communication unit 106 stores the receivedcommodity-of-interest information in the commodity-of-interest database102. In this case, the communication unit 106 stores thecommodity-of-interest information in association with the received userID.

As explained above, the commodity of interest is registered in theguidance service apparatus 100 in advance.

FIG. 4 is a first flowchart for explaining the operations of the guidingapparatus and the guidance service apparatus in the embodiment of thepresent invention. FIG. 5 is a second flowchart for explaining theoperations of the guiding apparatus and the guidance service apparatusin the embodiment of the present invention.

First, in step S11, the photographing unit 201 of the guiding apparatus200 photographs the periphery of the user in a store. In this case, thephotographing unit 201 photographs a shelf on which a plurality ofcommodities are displayed in the store.

Subsequently, in step S12, the present position acquiring unit 202acquires present position information indicating the present position ofthe guiding apparatus 200. The present position information isrepresented by, for example, latitude and longitude.

Subsequently, in step S13, the communication unit 303 transmits aphotographed image photographed by the photographing unit 201 and thepresent position information acquired by the present position acquiringunit 202 to the guidance service apparatus 100. In this case, thecommunication unit 303 adds the user ID for identifying the user to thephotographed image and the present position information and transmitsthe user ID. The user ID may be input by the user or may be registeredin the guiding apparatus 200 in advance.

Note that, in this embodiment, the guiding apparatus 200 acquires andtransmits the present position information. However, the presentinvention is not particularly limited to this. The guiding apparatus 200may receives an input of an address or a store name of the store andtransmits the received address or store name of the store as the presentposition information.

Subsequently, in step S14, the communication unit 106 of the guidanceservice apparatus 100 receives the photographed image and the presentposition information transmitted by the guiding apparatus 200.

Subsequently, in step S15, the commodity specifying unit 103 analyzesthe photographed image received by the communication unit 106 andspecifies a commodity included in the photographed image.

Subsequently, in step S16, the updating unit 105 specifies anarrangement position of the commodity specified by the commodityspecifying unit 103. In this case, the updating unit 105 analyzes thephotographed image to thereby recognize a shelf on which the specifiedcommodity is displayed, recognize on which plate of the shelf thespecified commodity is arranged, and specify on which plate of whichshelf in the store the specified commodity is arranged. Note that, inthe shelf on which the commodity is displayed in the store, for example,a shelf name and plate numbers are posted. The shelf name and the platenumbers are identified by an image analysis.

Subsequently, in step S17, the updating unit 105 determines whether thespecified arrangement position and an arrangement position correspondingto the commodity specified by the commodity specifying unit 103 acquiredfrom the commodity arrangement information retained in the commoditylayout database 101 are the same.

When it is determined that the specified arrangement position and theretained arrangement position are not the same (NO in step S17), in stepS18, the updating unit 105 updates the commodity layout database 101.That is, the updating unit 105 updates the arrangement positioncorresponding to the commodity specified by the commodity specifyingunit 103 retained in the commodity layout database 101 to the specifiedarrangement position.

Consequently, even if a new commodity is displayed or a commodity istransferred to another shelf, it is possible to always update thecommodity arrangement information of the commodity layout database 101to latest information. An administrator or the like does not need toupdate the commodity arrangement information of the commodity layoutdatabase 101. It is possible to automatically update the commodityarrangement information of the commodity layout database 101.

On the other hand, when it is determined that the specified arrangementposition and the retained arrangement position are the same (YES in stepS17), in step S19, the guidance information creating unit 104 determineswhether the commodity specified by the commodity specifying unit 103coincides with the commodity of interest retained in thecommodity-of-interest database 102.

When it is determined that the specified commodity coincides with thecommodity of interest (YES in step S19), in step S20, the guidanceinformation creating unit 104 creates, as the guidance information, animage indicating a position in the photographed image of the commodityof interest.

On the other hand, when it is determined that the specified commoditydoes not coincide with the commodity of interest (NO in step S19), instep S21, the guidance information creating unit 104 acquires anarrangement position corresponding to the specified commodity from thecommodity layout database 101.

Subsequently, in step S22, the guidance information creating unit 104acquires an arrangement position corresponding to the commodity ofinterest from the commodity layout database 101.

Subsequently, in step S23, the guidance information creating unit 104creates, as the guidance information, an image indicating the positionof the commodity of interest.

Subsequently, in step S24, the communication unit 106 transmits theguidance information created by the guidance information creating unit104 to the guiding apparatus 200.

Subsequently, in step S25, the communication unit 204 of the guidingapparatus 200 receives the guidance information transmitted by theguidance service apparatus 100.

Subsequently, in step S26, the display unit 302 displays the guidanceinformation received by the communication unit 204.

Subsequently, a specific display example of the guidance information isexplained. FIG. 6 is a diagram for explaining a photographing directionfor photographing the inside of a store by the user using the guidingapparatus in the embodiment of the present invention.

As shown in FIG. 6, a plurality of shelves are arranged in a store 400.Shelf names (H1 to H12) are allocated to the respective plurality ofshelves. The user performs photographing in a photographing direction401 indicated by an arrow in FIG. 6 using the guiding apparatus 200. Acommodity A is arranged on the shelf “H2”, a commodity B and a commodityD are arranged on the shelf “H3”, and a commodity C is arranged on theshelf “H5”. The commodity A, the commodity B, the commodity C, and thecommodity D are respectively commodities of interest.

FIG. 7 is a diagram showing an example of an in-store layout diagramretained in the commodity layout database. FIG. 8 is a diagram showingan example of commodity layout information retained in the commoditylayout database.

A in-store layout diagram 500 shown in FIG. 7 is a plan viewrepresenting the arrangement of the shelves in the store. Note that thein-store layout diagram 500 is two-dimensionally represented but may bethree-dimensionally represented. In commodity layout information 600shown in FIG. 8, a shelf name 610, a plate number 620, and a commodity630 are associated with one another. It is seen from the commoditylayout information 600 that, for example, the commodity A is arranged onthe second plate of the shelf name “H2”, the commodity B is arranged onthe first plate of the shelf name “H3”, the commodity C is arranged onthe first plate of the shelf name “H5”, and the commodity D is arrangedon the third plate of the shelf name “H3”.

When the specified commodity coincides with the commodity of interest inthe commodity-of-interest database 102 managed and stored by theguidance service apparatus 100, the guidance service apparatus 100notifies the guiding apparatus 200 of guidance information indicatingthat the commodity of interest is included in a photographing range. Theguiding apparatus 200 outputs guidance information indicating that thecommodity of interest is included in the photographing range.

FIG. 9 is a diagram showing an example of guidance information displayedwhen the commodity of interest is present in the photographed image.When the commodity A is the commodity of interest and photographed fromthe position shown in FIG. 6, the commodity of interest is present inthe photographed image. In this case, as shown in FIG. 9, the displayunit 203 of the guiding apparatus 200 presents guidance information 800indicating that the commodity of interest is present in front of theeyes. The guidance information 800 is represented in a text format.

FIG. 10 is a diagram showing another example of the guidance informationdisplayed when the commodity of interest is present in the photographedimage. FIG. 11 is a diagram showing an example of a user interfacecomponent image used when the guidance information shown in FIG. 10 iscreated.

The guidance information creating unit 104 creates guidance information810 in which an user interface component image 811 indicating theposition of the commodity of interest is superimposed on thephotographed image. Note that the user interface component image 811 isstored in advance in a memory of the guidance service apparatus 100.

Note that, in this embodiment, the guidance information creating unit104 creates the guidance information 810 in which the user interfacecomponent image 811 indicating the position of the commodity of interestis superimposed on the photographed image. However, the presentinvention is not particularly limited to this. The guidance informationcreating unit 104 may create, as the guidance information, only the userinterface component image 811 indicating the position of the commodityof interest. In this case, the guiding apparatus 200 superimposes theguidance information (the user interface component image 811) receivedfrom the guidance service apparatus 100 on an image photographed by thephotographing unit 201 on a real time basis and displays the guidanceinformation. The guidance information is transmitted together withinformation for specifying a display position of the guidanceinformation on a display screen of the display unit 203.

When the commodity of interest is absent among commodities specifiedfrom the image photographed by the guiding apparatus 200, the guidanceservice apparatus 100 creates, using the commodity layout database 101,guidance information for guiding the user to the commodity of interest.

FIG. 12 is a diagram showing an example of guidance informationdisplayed when the commodity of interest is absent in the photographedimage.

When the commodity of interest is absent among the commodities specifiedfrom the image photographed by the guiding apparatus 200, the guidanceinformation creating unit 104 of the guidance service apparatus 100accesses the commodity layout database 101 and acquires the position ofa commodity specified from the photographed image in the store. Forexample, when the commodity A is the specified commodity, by referringto the commodity layout database 101, the guidance information creatingunit 104 can specify that the commodity A is present on the second plateof the shelf H2.

Subsequently, the guidance information creating unit 104 accesses thecommodity layout database 101 and acquires a position in the store ofthe commodity of interest. For example, when the commodity B is thecommodity of interest, by referring to the commodity layout database101, the guidance information creating unit 104 can specify that thecommodity B is present on the first plate of the shelf H3.

Subsequently, the guidance information creating unit 104 createsguidance information for guiding the user from the position in the storeof the commodity specified from the acquired photographed image to theposition in the store of the acquired commodity of interest.

The guidance service apparatus 100 notifies the guiding apparatus 200 ofthe guidance information for guiding the user from the photographingposition to the position of the commodity of interest. The guidingapparatus 200 outputs the guidance information for guiding the user fromthe photographing position to the position of the commodity of interest.

Note that the guidance service apparatus 100 may notify the guidingapparatus 200 of the position of the commodity of interest. The guidingapparatus 200 may create, on the basis of the position of the commodityof interest notified by the guidance service apparatus 100, the guidanceinformation for guiding the user from the photographing position to theposition of the commodity of interest.

When the commodity B is present in the commodity-of-interest list andthe commodity A is the specified commodity, the guidance informationcreating unit 104 can specify that the commodity B is displayed on thefirst plate of the first shelf in the right direction from the commodityA.

In this case, as shown in FIG. 12, the display unit 203 of the guidingapparatus 200 presents guidance information 820 indicating in a textformat that the commodity B is present on the top plate of the firstshelf in the right direction.

FIG. 13 is a diagram showing another example of the guidance informationdisplayed when the commodity of interest is absent in the photographedimage. FIG. 14 is a diagram showing an example of a user interfacecomponent image used when the guidance information shown in FIG. 13 iscreated.

The guidance information creating unit 104 creates guidance information830 in which a user interface component image 831 indicating theposition of the commodity of interest is superimposed on thephotographed image. The guidance information 830 indicates a directionfrom the position of the specified commodity to the position where thecommodity of interest is present. Note that the user interface componentimage 831 is stored in the memory in the guidance service apparatus 100in advance.

When the commodity B is present in the commodity-of-interest list andthe commodity A is the specified commodity, as shown in FIG. 13, thedisplay unit 203 of the guiding apparatus 200 presents the guidanceinformation 830 indicating that the commodity B is present in the rightdirection.

As shown in FIG. 13, the display unit 203 of the guiding apparatus 200may superimpose a relative position of the commodity of interest B on anactually photographed image and present the relative position. Note thatthe user interface component image 831 is an example of an imagesuperimposed on the actually photographed image.

Note that, in this embodiment, the guidance information creating unit104 creates the guidance information 830 in which the user interfacecomponent image 831 indicating the position of the commodity of interestis superimposed on the photographed image. However, the presentinvention is not particularly limited to this. The guidance informationcreating unit 104 may create, as the guidance information, only the userinterface component image 831 indicating the position of the commodityof interest. In this case, the guiding apparatus 200 superimposes theguidance information (the user interface component image 831) receivedfrom the guidance service apparatus 100 on an image photographed by thephotographing unit 201 on a real time basis and displays the guidanceinformation. The guidance information is transmitted together withinformation for specifying a display position of the guidanceinformation on the display screen of the display unit 203.

When the commodity of interest is present in an invisible position inthe photographed image, the guiding apparatus 200 may show an imaginaryposition of the commodity of interest.

FIG. 15 is a diagram showing an example of the guidance informationdisplayed when the commodity of interest is present in an invisibleposition in the photographed image. FIG. 16 is a diagram showing anexample of a user interface component image used when the guidanceinformation shown in FIG. 15 is created.

For example, the commodity C is present in the commodity-of-interestlist, the commodity C is arranged on a shelf on the rear side of a shelfon which the commodity A specified from the photographed image isarranged, and the commodity C is blocked by the shelf on which thecommodity A is displayed and cannot be directly photographed but ispresent within a field of view of the photographed image.

In this case, as shown in FIG. 15, the display unit 203 of the guidingapparatus 200 may superimpose an imaginary position in athree-dimensional space of the commodity of interest C on an actuallyphotographed image and present the imaginary position. Note that a userinterface component image 841 is an example of an image superimposed onthe actually photographed image.

The guidance information creating unit 104 creates guidance information840 in which the user interface component image 841 indicating animaginary position in a three-dimensional space of the commodity ofinterest is superimposed on the photographed image. The guidanceinformation 840 indicates the imaginary position where the commodity ofinterest is present in the three-dimensional space. Note that the userinterface component image 841 is stored in the memory in the guidanceservice apparatus 100 in advance.

When the commodity C is present in the commodity-of-interest list, thecommodity A is the specified component, and the commodity C is displayedon a shelf on the rear side of the shelf on which the commodity A isdisplayed, as shown in FIG. 15, the display unit 203 of the guidingapparatus 200 presents the guidance information 840 indicating that thecommodity C is present on the shelf on the rear side of the shelf onwhich the commodity A is displayed. In this case, an invisible shelfimage is actually superimposed on the photographed image together withthe position of the commodity C, which is the commodity of interest.

Note that, in this embodiment, the guidance information creating unit104 creates the guidance information 840 in which the user interfacecomponent image 841 indicating the imaginary position of the commodityof interest is superimposed on the photographed image. However, thepresent invention is not particularly limited to this. The guidanceinformation creating unit 104 may create, as the guidance information,only the user interface component image 841 indicating an imaginaryposition of the commodity of interest. In this case, the guidingapparatus 200 superimposes the guidance information (the user interfacecomponent image 841) received from the guidance service apparatus 100 onan image photographed by the photographing unit 201 on a real time basisand displays the guidance information. The guidance information istransmitted together with information for specifying a display positionof the guidance information on the display screen of the display unit203.

FIG. 17 is a diagram showing an example of guidance informationdisplayed when a specified commodity is not a commodity of interest andthe commodity of interest is present in a photographed image. FIG. 18 isa diagram showing an example of a user interface component image usedwhen the guidance information shown in FIG. 17 is created.

In the example shown in FIG. 17 and FIG. 18, even when the resolution ofthe photographed image is low and the commodity of interest in thephotographed image is not specified as a commodity, the guidingapparatus 200 presents the position of the commodity of interest in thephotographed image.

That is, when it is determined that the specified commodity does notcoincide with the commodity of interest, the guidance informationcreating unit 104 acquires an arrangement position corresponding to thespecified commodity. Subsequently, the guidance information creatingunit 104 acquires an arrangement position corresponding to the commodityof interest. Subsequently, the guidance information creating unit 104specifies the position of a shelf included in the photographed image anddetermines whether the acquired arrangement position corresponding tothe commodity of interest is included in the photographed image. Notethat the position of the shelf included in the photographed image can bespecified by identifying a shelf name posted on the shelf through animage analysis.

When the arrangement position corresponding to the commodity of interestis included in the photographed image, the guidance information creatingunit 104 creates, as the guidance information, an image indicating aposition in the photographed image of the commodity of interest. On theother hand, when the arrangement position corresponding to the commodityof interest is not included in the photographed image, the guidanceinformation creating unit 104 creates, as the guidance information, animage indicating the position of the commodity of interest (an imageindicating a direction in which the commodity of interest is present).

When the arrangement position corresponding to the commodity of interestis included in the photographed image, the guidance information creatingunit 104 creates guidance information 850 in which a user interfacecomponent image 851 indicating the position of the commodity of interestis superimposed on the photographed image. The guidance information 850indicates the position where the commodity of interest is present. Notethat the user interface component image 851 is stored in the memory ofthe guidance service apparatus 100 in advance.

When the commodity B is present in the commodity-of-interest list andthe commodity A is the specified commodity, as shown in FIG. 17, thedisplay unit 203 of the guiding apparatus 200 presents the guidanceinformation 850 indicating that the commodity of interest B is presentin the photographed image.

As shown in FIG. 17, the display unit 203 of the guiding apparatus 200may superimpose a relative position of the commodity of interest B on anactually photographed image and present the relative position. Note thatthe user interface component image 851 is an example of an imagesuperimposed on the actually photographed image.

Note that, in this embodiment, the guidance information creating unit104 creates the guidance information 850 in which the user interfacecomponent image 851 indicating the position of the commodity of interestis superimposed on the photographed image. However, the presentinvention is not particularly limited to this. The guidance informationcreating unit 104 may creates, as the guidance information, only theuser interface component image 851 indicating the position of thecommodity of interest. In this case, the guiding apparatus 200superimposes the guidance information (the user interface componentimage 851) received from the guidance service apparatus 100 on an imagephotographed by the photographing unit 201 on a real time basis anddisplays the guidance information. The guidance information istransmitted together with information for specifying a display positionof the guidance information on the display screen of the display unit203.

FIG. 19 is a diagram showing an example of guidance informationdisplayed when a commodity of interest is absent in a photographed imagein which commodities are two-dimensionally arranged. FIG. 20 is adiagram showing an example of a user interface component image used whenthe guidance information shown in FIG. 19 is created.

The guidance information creating unit 104 creates guidance information860 in which a user interface component image 861 indicating theposition of the commodity of interest is superimposed on a photographedimage in which commodities are two-dimensionally arranged. The guidanceinformation 860 indicates a direction from the position of a specifiedcommodity to the position where the commodity of interest is present.Note that the user interface component image 861 is stored in the memoryin the guidance service apparatus 100 in advance.

When the commodity D is present in the commodity-of-interest list andthe commodity B is the specified commodity, as shown in FIG. 19, thedisplay unit 203 of the guiding apparatus 200 presents the guidanceinformation 860 indicating that the commodity D is present in thedownward direction.

As shown in FIG. 19, the display unit 203 of the guiding apparatus 200may superimpose a relative position of the commodity of interest D on anactually photographed image and present the relative position. Note thatthe user interface component image 861 is an example of an imagesuperimposed on the actually photographed image.

Note that, in this embodiment, the guidance information creating unit104 creates the guidance information 860 in which the user interfacecomponent image 861 indicating the position of the commodity of interestis superimposed on the photographed image. However, the presentinvention is not particularly limited to this. The guidance informationcreating unit 104 may create, as the guidance information, only the userinterface component image 861 indicating the position of the commodityof interest. In this case, the guiding apparatus 200 superimposes theguidance information (the user interface component image 861) receivedfrom the guidance service apparatus 100 on an image photographed by thephotographing unit 201 on a real time basis and displays the guidanceinformation. The guidance information is transmitted together withinformation for specifying a display position of the guidanceinformation on the display screen of the display unit 203.

Note that, in this embodiment, the commodity of interest registered inthe commodity-of-interest database 102 is not limited to one commodityof interest. A plurality of commodities of interest are sometimesregistered. Therefore, when arrangement positions of the plurality ofcommodities of interest are acquired, the guidance information creatingunit 104 may create guidance information for guiding the user to acommodity of interest present in a position closest from a specifiedcommodity. When the arrangement positions of the plurality ofcommodities of interest are acquired, the guidance information creatingunit 104 may create guidance information for guiding the user to acommodity of interest recently registered. Further, when the arrangementpositions of the plurality of commodities of interest are acquired, theguidance information creating unit 104 may create guidance informationfor guiding the user to the commodities of interest in order from acommodity of interest present in a position closest from the specifiedcommodity. Furthermore, when the arrangement positions of the pluralityof commodities of interest are acquired, the guidance informationcreating unit 104 may create guidance information for guiding the userto the commodities of interest in order from a commodity of interestrecently registered.

In this embodiment, the guidance information is displayed by the displayunit 203 included in the guiding apparatus 200. However, the presentinvention is not particularly limited to this. The guidance informationmay be output by sound by a sound output unit included in the guidingapparatus 200. The guiding apparatus 200 may include a vibrating devicethat vibrates the guiding apparatus 200. The vibrating device mayvibrate the guiding apparatus 200 such that the user moves in thedirection of the commodity of interest represented by the guidanceinformation.

In this embodiment, when a plurality of kinds of commodityidentification information in a photographed image are specified by thecommodity specifying unit 103, the guidance information creating unit104 may specify arrangement positions in the building of the commoditiesof interest on the basis of one kind of commodity identificationinformation among the specified plurality of kinds of commodityidentification information, the commodity arrangement informationretained in the commodity layout database 101, and thecommodity-of-interest identification information registered by the userin advance. For example, commodity identification informationcorresponding to a commodity closest to the center of the photographedimage is selected as one kind of commodity identification informationamong the specified plurality of kinds of commodity identificationinformation.

Note that inventions including configurations explained below are mainlyincluded in the specific embodiment explained above.

A guiding method according to an aspect of the present inventionincludes: a commodity arrangement information retaining step ofretaining, in a commodity arrangement information retaining unit,commodity arrangement information in which a plurality of kinds ofcommodity identification information for identifying a plurality ofcommodities arranged indoors and indoor arrangement positions of theplurality of commodities are associated with each other; a photographingstep of photographing the periphery of an indoor user; a specifying stepof analyzing a photographed image photographed in the photographing stepand specifying commodity identification information included in thephotographed image; and a guiding step of specifying an indoorarrangement position of a registered commodity registered in advance bythe user on the basis of the commodity identification informationspecified in the specifying step, the commodity arrangement informationretained in the commodity arrangement information retaining unit, andregistered commodity identification information for identifying theregistered commodity, and presenting guidance information for guidingthe user to the specified arrangement position of the registeredcommodity.

With this configuration, the commodity arrangement information isretained in which the plurality of kinds of commodity identificationinformation for identifying the plurality of commodities arrangedindoors and the indoor arrangement positions of the plurality ofcommodities are associated with each other. The commodity identificationinformation included in the photographed image is specified. The indoorarrangement position of the registered commodity is specified on thebasis of the specified commodity identification information, thecommodity arrangement information, and the registered commodityidentification information for identifying a registered commodityregistered in advance by the user. The guidance information for guidingthe user to the specified arrangement position of the registeredcommodity is presented.

Therefore, it is possible to guide the user to an indoor position of acommodity in which the user is interested using arrangement positions ofcommodities included in the photographed image and an indoor arrangementposition of the registered commodity registered by the user in advance.

In the guiding method, it is preferable that, in the guiding step, whenthe commodity identification information specified in the specifyingstep and the registered commodity identification information coincidewith each other, an image indicating a position in the photographedimage of the registered commodity is presented as the guidanceinformation.

With this configuration, when the commodity identification informationincluded in the photographed image and the registered commodityidentification information coincide with each other, the imageindicating the position in the photographed image of the registeredcommodity is presented as the guidance information. Therefore, when theregistered commodity is included in the photographed image, it ispossible to present the position in the photographed image of theregistered commodity to the user.

In the guiding method, it is referable that, in the guiding step, whenthe commodity identification information specified in the specifyingstep and the registered commodity identification information do notcoincide with each other, an image indicating the position of theregistered commodity is presented as the guidance information.

With this configuration, when the specified commodity identificationinformation and the registered commodity identification information donot coincide with each other, the image indicating the position of theregistered commodity is presented as the guidance information.Therefore, even when the registered commodity is not included in thephotographed image, it is possible to present the position of theregistered commodity to the user.

In the guiding method, it is preferable that, in the guiding step, whenthe commodity identification information specified in the specifyingstep and the registered commodity identification information do notcoincide with each other, an image indicating a direction in which theregistered commodity is present is presented as the guidanceinformation.

With this configuration, when the specified commodity identificationinformation and the registered commodity identification information donot coincide with each other, the image indicating the direction inwhich the registered commodity is present is presented to the user asthe guidance information. Therefore, even when the registered commodityis not included in the photographed image, it is possible to present thedirection in which the registered commodity is present to the user.

In the guiding method, it is preferable that, in the guiding step, whenthe commodity identification information specified in the specifyingstep and the registered commodity identification information do notcoincide with each other and the registered commodity is present in ahidden position in the photographed image, an image indicating animaginary position of the registered commodity is presented as theguidance information.

With this configuration, when the specified commodity identificationinformation and the registered commodity identification information donot coincide with each other and the registered commodity is present inthe hidden position in the photographed image, the image indicating theimaginary position of the registered commodity is presented as theguidance information. Therefore, even when the registered commodity ispresent in the hidden position in the photographed image, it is possibleto present the imaginary position of the registered commodity to theuser.

It is preferable that the guiding method further includes an updatingstep of specifying an indoor arrangement position of the commodityidentification information specified in the specifying step and, whenthe specified arrangement position is different from an arrangementposition of the commodity identification information included in thecommodity arrangement information, updating the commodity arrangementinformation to change the arrangement position of the commodityidentification information included in the commodity arrangementinformation to the specified arrangement information.

With this configuration, the commodity arrangement information isupdated on the basis of the photographed image. Therefore, it ispossible to update the commodity arrangement information to a lateststate every time a photographed image is acquired. It is possible topresent the position of the registered commodity to the user using thelatest commodity arrangement information.

In the guiding method, it is preferable that the registered commodityincludes a plurality of registered commodities, and, in the guidingstep, guidance information for guiding the user to an arrangementposition of a registered commodity closest to the arrangement positionof the commodity identification information specified in the specifyingstep among the plurality of registered commodities is presented.

With this configuration, the guidance information for guiding the userto the arrangement position of the registered commodity closest to thespecified arrangement position of the commodity identificationinformation among the plurality of registered commodities is presented.Therefore, even if a plurality of commodities are registered, it ispossible to guide the user to a registered commodity present in aposition closest from the present position of the user.

A guiding system according to another aspect of the present invention isa guiding system including: a server apparatus; and a guiding apparatuscommunicably connected to the server apparatus. The server apparatusincludes: a commodity arrangement information retaining unit thatretains commodity arrangement information in which a plurality of kindsof commodity identification information for identifying a plurality ofcommodities arranged indoors and indoor arrangement positions of theplurality of commodities are associated with each other; a photographedimage acquiring unit that acquires a photographed image obtained byphotographing the periphery of an indoor user; a specifying unit thatanalyzes the photographed image acquired by the photographed imageacquiring unit and specifies commodity identification informationincluded in the photographed image; and a guidance information creatingunit that specifies an indoor arrangement position of a registeredcommodity registered in advance by the user on the basis of thecommodity identification information specified by the specifying unit,the commodity arrangement information retained by the commodityarrangement information retaining unit, and registered commodityidentification information for identifying the registered commodity, andcreates guidance information for guiding the user to the specifiedarrangement position of the registered commodity. The guiding apparatusincludes: a photographing unit that photographs the periphery of theindoor user; and a guidance information presenting unit that presentsthe guidance information created by the guidance information creatingunit.

With this configuration, in the server apparatus, the commodityarrangement information is retained in which the plurality of kinds ofcommodity identification information for identifying the plurality ofcommodities arranged indoors and the indoor arrangement positions of theplurality of commodities are associated with each other. Thephotographed image obtained by photographing the periphery of the indooruser is acquired. The acquired photographed image is analyzed and thecommodity identification information included in the photographed imageis specified. The indoor arrangement position of the registeredcommodity is specified on the basis of the specified commodityidentification information, the retained commodity arrangementinformation, and the registered commodity identification information foridentifying the registered commodity registered in advance by the user.The guidance information for guiding the user to the specifiedarrangement position of the registered commodity is created. In theguiding apparatus, the periphery of the indoor user is photographed. Thecreated guidance information is presented.

Therefore, it is possible to guide the user to an indoor position of acommodity in which the user is interested using arrangement positions ofcommodities included in the photographed image and an indoor arrangementposition of the registered commodity registered by the user in advance.

Note that the specific embodiment or example explained in thedescription of embodiments are solely for clarifying the technicalcontents of the present invention. The specific embodiment or exampleshould not be interpreted in a narrow sense to be limited to only suchspecific examples and can be variously changed and carried out withinthe spirit of the present invention and the scope of the claimedmatters.

INDUSTRIAL APPLICABILITY

The guiding method and the guiding system according to the presentinvention can guide a user to the indoor position of a commodity ofinterest and is useful for a guiding method and a guiding system thatguide the user to the position of the commodity of interest.

1. A guiding method comprising: storing, in a memory of a serverapparatus, commodity arrangement information in which: (i) a pluralityof commodity identification information that identifies a respectiveplurality of commodities, and (ii) respective arrangement positions ofthe plurality of commodities, are associated with each other;registering, with the server apparatus, a commodity of interest, thecommodity of interest having commodity identification information andassociated position information in the commodity arrangement informationstored in the memory of the server apparatus; obtaining, with the serverapparatus, from a user device at least one image, the at least one imagebeing generated by taking a photograph with the user device from aperspective of the user; obtaining, with the server apparatus, commodityidentification information that is included in the image, wherein theobtained commodity identification information is associated with acommodity that is visible in the image; finding, in the commodityarrangement information in the memory of the server apparatus, theassociated position information of the commodity that is visible in theimage using the obtained commodity identification information that isincluded in the image; obtaining, from the memory of the serverapparatus, the associated position information of the commodity ofinterest; when the commodity identification information that is includedin the image and the commodity identification information of thecommodity of interest do not coincide with each other and the commodityof interest is present in a hidden position of the image, creating, withthe server apparatus, guidance information for guiding the user to thehidden position of the commodity of interest, the guidance informationbeing obtained by superimposing an interface image indicating the hiddenposition of the commodity of interest on the image in such a manner asto display the hidden position of the commodity of interest on the imageassociated with the commodity that is visible in the image; and sending,with the server apparatus, the created guidance information to the userdevice.
 2. A guiding method comprising: storing, in a memory of a serverapparatus, commodity arrangement information in which: (i) a pluralityof commodity identification information that identifies a respectiveplurality of commodities, and (ii) respective arrangement positions ofthe plurality of commodities, are associated with each other;registering, with the server apparatus, a commodity of interest, thecommodity of interest having commodity identification information andassociated position information in the commodity arrangement informationstored in the memory of the server apparatus; obtaining, with the serverapparatus, from a user device at least one image, the at least one imagebeing generated by taking a photograph with the user device from aperspective of the user; obtaining, with the server apparatus, commodityidentification information that is included in the image, wherein theobtained commodity identification information is associated with acommodity that is visible in the image; finding, in the commodityarrangement information in the memory of the server apparatus, theassociated position information of the commodity that is visible in theimage using the obtained commodity identification information that isincluded in the image; obtaining, from the memory of the serverapparatus, the associated position information of the commodity ofinterest; when the commodity identification information that is includedin the image and the commodity identification information of thecommodity of interest do not coincide with each other and the commodityof interest is not present in the image, creating, with the serverapparatus, guidance information for guiding the user to a relativeposition of the commodity of interest, the guidance information beingobtained by superimposing an interface image indicating the relativeposition of the commodity of interest on the image in such a manner asto display the relative position of the commodity of interest on theimage associated with the commodity; and sending, with the serverapparatus, the created guidance information to the user device.